Skip to content

topology2: add WebRTC HPF/AGC capture chaining, HDA AEC, and PTL targets - #11426

Draft
lgirdwood wants to merge 10 commits into
thesofproject:mainfrom
lgirdwood:topic/webrtc-pr2
Draft

lgirdwood wants to merge 10 commits into
thesofproject:mainfrom
lgirdwood:topic/webrtc-pr2

Conversation

@lgirdwood

Copy link
Copy Markdown
Member

Summary

Follow-up topology2 improvements for the WebRTC audio processing module suite (#11044, #11419):

  1. topology2: add missing direction attribute to capture pipelines

    • Add explicit direction "capture" to the pipeline."1" scheduler widget in dai-copier-eqiir-module-copier-capture.conf, host-gateway-capture.conf, and webrtc-{aec,ns,ns2}-capture.conf so the pipeline widget direction matches the enclosing capture pipeline class.
  2. topology2: cavs-mixin-mixout-hda: add WebRTC AEC capture support

    • Add an optional AEC_PROCESSING flag to cavs-mixin-mixout-hda.conf that inserts webrtc-aec.3.1 between module-copier.4.2 and host-copier.0.capture on the analog capture path, and register sof-hda-generic-webrtc-aec in development/tplg-targets.cmake.
  3. topology2: wire webrtc-hpf and webrtc-agc into capture and add PTL

    • Chain webrtc-hpf and webrtc-agc into webrtc-ns-capture.conf (dai-copier -> webrtc-hpf -> webrtc-vad -> webrtc-ns -> webrtc-agc -> host-copier) and add webrtc-hpf pre-filtering ahead of webrtc-ns2 in webrtc-ns2-capture.conf.
    • Add Panther Lake (PTL / ACE 3.0) development topology targets sof-ptl-nocodec-webrtc-{ns,ns2,aec} in development/tplg-targets.cmake.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The HDA AEC lacks a playback-reference input needed to cancel speaker echo, and its default suppression control name exceeds ALSA topology’s 43-byte name limit.

2 open issues
What changed in this PR

This PR extends topology2’s WebRTC capture processing and adds development targets for HDA and Panther Lake.

Changes:

  • Sets explicit capture direction on five pipeline scheduler widgets.
  • Adds HPF and AGC stages to the existing VAD/NS capture chain.
  • Adds an optional HDA AEC capture path and registers HDA AEC and PTL WebRTC targets.
File Description
tools/​topology/​topology2/​include/​pipelines/​cavs/​webrtc-ns2-capture.conf Sets capture scheduler direction.
tools/​topology/​topology2/​include/​pipelines/​cavs/​webrtc-ns-capture.conf Adds HPF/AGC widgets and routes; sets capture direction.
tools/​topology/​topology2/​include/​pipelines/​cavs/​webrtc-aec-capture.conf Sets capture scheduler direction.
tools/​topology/​topology2/​include/​pipelines/​cavs/​host-gateway-capture.conf Sets capture scheduler direction.
tools/​topology/​topology2/​include/​pipelines/​cavs/​dai-copier-eqiir-module-copier-capture.conf Sets capture scheduler direction.
tools/​topology/​topology2/​development/​tplg-targets.cmake Registers PTL NS, NS2, and AEC targets plus HDA AEC.
tools/​topology/​topology2/​development/​cavs-nocodec-webrtc-ns.conf Updates comments and diagram for the expanded capture chain.
tools/​topology/​topology2/​cavs-mixin-mixout-hda.conf Adds optional AEC widget, controls, and analog capture routing.

Review effort: Balanced

@lgirdwood lgirdwood left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the monolithic topology support, can we also look at adding these modules as feature or function topologies.

@lgirdwood

Copy link
Copy Markdown
Member Author

@lgirdwood Added modular WebRTC feature and function topologies in commit 199d63076 (topology2: add WebRTC feature and function topologies for DMIC/SDW):

  • Function topology fragment:
    • tools/topology/topology2/platform/intel/dmic1-webrtc.conf: DMIC PCM 1 capture function fragment (WEBRTC_FUNCTION="ns" | "ns2", wiring dai-copier.DMIC -> module-copier -> webrtc-hpf -> webrtc-ns/ns2 -> webrtc-agc -> webrtc-vad -> host-copier), wired into cavs-sdw.conf and platform/intel/dmic-generic.conf via WEBRTC_FUNCTION.
  • Feature topology wrappers:
    • tools/topology/topology2/dmic-webrtc-feature.conf: standalone DMIC feature topologies (sof-dmic-2ch-webrtc-ns, sof-dmic-2ch-webrtc-ns2).
    • tools/topology/topology2/platform/intel/sdw-dmic-webrtc-feature.conf: SoundWire SDCA microphone feature topologies (sof-sdca-mic-id4-webrtc-ns, sof-sdca-mic-id4-webrtc-ns2).
    • tools/topology/topology2/platform/intel/sdw-jack-webrtc-feature.conf: SoundWire SDCA jack feature topologies (sof-sdca-jack-id0-webrtc-ns, sof-sdca-jack-id0-webrtc-aec with playback echo-reference tap from module-copier.1.2).
    • Composite development target sof-ptl-dmic-2ch-id2-webrtc-ns in development/tplg-targets.cmake.

All 14 WebRTC topologies compile cleanly with alsatplg and pass tplg_validate.py with 0 errors. Also tested the updated firmware, LLEXTs, and sof-hda-generic-webrtc-aec.tplg on damsel-fly (PTL / ACE 3.0) across capture-only, playback-only, simultaneous playback+capture (2-source AEC), and dynamic mid-stream playback start/stop.

Comment thread src/audio/webrtc_aec/webrtc_aec-shims.c Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical defects in userspace notification dispatch, logarithm calculations, and capture topology formats and pipeline IDs risk faults, incorrect audio processing, and incompatible capture paths.

7 open issues
2 resolved since last review

Review effort: Balanced

Comment thread src/audio/webrtc_ns/webrtc_ns-shims.c Resolved
Comment thread src/lib/notifier.c Resolved
Add explicit direction "capture" to the pipeline."1" scheduler widget in
dai-copier-eqiir-module-copier-capture.conf, host-gateway-capture.conf,
and webrtc-{aec,ns,ns2}-capture.conf so the pipeline widget direction
matches the enclosing capture pipeline class.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Instantiate webrtc-hpf (DC/<80 Hz removal) and webrtc-agc (digital gain
control and saturation limiter) in webrtc-ns-capture.conf alongside
webrtc-vad and webrtc-ns to form the complete WebRTC voice
pre-processing chain.

Add PTL nocodec topology targets (sof-ptl-nocodec-webrtc-{ns,aec,ns2})
in tplg-targets.cmake alongside the existing TGL targets.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add an optional WebRTC AEC capture variant in cavs-mixin-mixout-hda.conf
using webrtc-aec-capture, tap the analog playback pipeline after drc.2.1
via module-copier.2.2 to feed the speaker reference signal to
webrtc-aec.3.1 input pin 1, and keep the AEC mixer control names within
the 44-byte ALSA control name limit.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add WebRTC NS/NS2/AEC topology targets and platform/intel configs for
DMIC, SDW DMIC, and SDW Jack capture chains (including pipeline ID 450
for SDW Jack WebRTC capture, explicit module-copier S32->S16 conversion
after 48->16 kHz SRC, and matching 16 kHz / DMIC1_RATE format sets on
webrtc-vad).

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Compile external WebRTC static libraries (AECm, NS, RNNoise, libfvad) with
-fno-pic -fno-pie -mlongcalls -mtext-section-literals -DNDEBUG when
building relocatable Xtensa LLEXTs. Without these flags, GCC emits
R_XTENSA_PLT internal relocations that the Zephyr LLEXT loader cannot
resolve against static archive symbols.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Zephyr sets LL_EXTENSION_BUILD rather than LLEXT_MODULE when compiling
relocatable LLEXT sources; check both macros in the WebRTC shim files so
malloc/free wrappers and non-userspace BSS placement match the build
mode. Also remove the unused <rtos/panic.h> include and k_panic() call
from webrtc_aec-shims.c.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Provide __attribute__((weak)) soft-float logf, log10f, expf, cosf, sinf,
and atanf shims compiled unconditionally so both relocatable LLEXT and
monolithic builds (e.g. CAVS 2.5 where CONFIG_LIBRARY_MANAGER=n) link
without an external libm or duplicate symbol collisions. Implement
log2f and exp2f via range reduction into [sqrt(2)/2, sqrt(2)] and
[-0.5, 0.5] so inputs across the full positive domain stay accurate and
continuous.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
When CONFIG_USERSPACE is enabled, webrtc_vad runs in a user thread and
invokes notifier_event() on VAD state changes. Instead of exporting raw
notifier_event() directly to LLEXT/userspace (where host_notify state is
not partitioned for user access), expose notifier_event() via a Zephyr
__syscall wrapper and validate the event type and caller buffer in
z_vrfy_notifier_event().

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Set init_config = "1" in webrtc_aec.toml so topology generates the ext
init_config metadata expected by IPC4 module setup, and guard
input_pins[1] access with nb_input_pins > 1 && input_pins != NULL in
webrtc_aec_prepare() so single-source preparation without a second pin
descriptor cannot dereference past input_pins.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Re-resolve mic and playback-reference sources in webrtc_aec_process()
and webrtc_aec_is_ready_to_process() because IPC4 playback pipelines may
bind or unbind dynamically while capture is active, and drop excess
reference frames so playback backpressure does not stall capture or overflow
the reference ring buffer.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants